1. Field of the Invention
The present invention relates to an image retrieval information storing apparatus and an image retrieving apparatus, retrieving an image based on the image retrieval information. More specifically, the present invention relates to an image retrieval information storing apparatus and an image retrieving apparatus, using coding information utilized for featured frame extraction or feature values of respective frames, as image retrieval information.
2. Description of the Background Art
When a user retrieves a motion picture, it is important that the user recognizes the contents of the motion picture, that is, what is the object picked up in the motion picture. Except when the motion picture has a text title or description, in most cases it is necessary for the user to confirm the contents by playing the whole motion picture. As the number of motion pictures among which a desired one is to be retrieved increases, it becomes harder for the user to play the motion pictures one by one and to confirm the contents thereof, and the efficiency of retrieval is quite poor. The efficiency of retrieval further lowers when each motion picture is in the form of coded compressed data requiring decoding and reproduction before retrieval. In order to solve this problem, a method of retrieval may be effective in which one or a plurality of frames characteristic of the motion picture, typically representing the contents of the motion picture are extracted and displayed, so that the contents of the motion picture can be displayed in schematic manner.
The characteristic frame (hereinafter referred to as featured frame) used for motion picture retrieval includes, for example, a scene change frame. Japanese Patent Laying-Open No. 9-261648 discloses a method in which a scene change frame is extracted, based on prediction mode information included in image data, from image data coded in accordance with interframe prediction coding. Over one frame, the number of I blocks which are coded in accordance with intraframe prediction, P blocks coded in accordance with forward interframe prediction, B blocks coded in accordance with backward interframe prediction and Bi blocks coded in accordance with bidirectional interframe prediction are counted. Frequency information of respective blocks reflects correlation among frames in the motion picture. Thus, based on the frequency information of each block, a frame immediately following a scene change (= scene change frame) is detected. Further, Japanese Patent Laying-Open No. 10-23421 discloses a method of extracting a scene change frame, based on the change in amount of each piece of coding information contained in the coded image data. A conventional image retrieval information storing apparatus in accordance with the prior art is as shown in FIG. 14.
Referring to FIG. 14, the conventional image retrieval information storing apparatus includes a coding information reading unit 1401, a frame feature value generating unit 1402 connected to coding information reading unit 1401, an index information generating unit 1403 connected to frame feature value generating unit 1402, and an index information storing unit 1404 connected to index information generating unit 1403.
Coding information reading unit 1401 reads, according to Japanese Patent Laying-Open No. 9-261648, block by block prediction mode information from coded image data, and reads, according to Japanese Patent Laying-Open No. 10-23421, respective pieces of coding information of different types. Frame feature value generating unit 1402 calculates, from the various pieces of coding information read by coding information reading unit 1401, frequency of respective prediction mode blocks according to Japanese Patent Laying-Open No. 9-261648, and calculates accumulated value of amount of respective pieces of coding information according to Japanese Patent Laying-Open No. 10-23421. Index information generating unit 1403 determines the scene change frame which is the featured frame, based on the frame by frame feature value obtained in frame feature value generating unit 1402, that is, frequency information of respective prediction mode blocks according to Japanese Patent Laying-Open No. 9-261648, and accumulated amount of respective pieces of coding information according to Japanese Patent Laying-Open No. 10-23421. Index information generating unit 1403 outputs positional information of the scene change frame as index information. Index information storing unit 1404 stores the index information output from the index information generating unit 1403 as image retrieval information.
FIG. 15 represents a conventional image retrieving apparatus using the image retrieval information stored in this manner. Referring to FIG. 15, the conventional image retrieving apparatus includes an image retrieval executing unit 1501 executing retrieval using the image retrieval information and the coded image data as inputs and outputting the results of retrieval.
The featured frame includes, in addition to the scene change frame, a scene change frame based on rapid/slow movement, disclosed in Japanese Patent Laying-Open No. 9-284702. According to the invention disclosed in Japanese Patent Laying-Open No. 9-284702, a motion vector for detecting a scene change frame is obtained from the original image, not from the coded information, and the scene change frame is detected based on the total sum of motion vectors in a frame.
In the above described conventional image retrieval information storing apparatus, positional information of the extracted featured frame is obtained, and the positional information only is stored as the index information. Therefore, even when a plurality of images which are the objects of retrieval have mutually different features, only a featured frame representing one same feature only can be obtained. If feature frames reflecting features of respective images are to be obtained, it is necessary for the user to again generate the image retrieval information from the coded image data. In the process of retrieval, when featured frames corresponding to various features of images which are the object of retrieval are to be taken out in an interactive manner, re-generation of the image retrieval information from the coded image data significantly lowers retrieval efficiency. Further, preparation of featured frame indexes corresponding to every possible features for all the image data is not practical. Further, it is not possible by the prior art to meet with high efficiency a request for extracting a featured frame which has not been expected by the provider providing the image retrieval information.